Program synthesis with existentially and universally quantified belief propagation using probabilistic inference

ABSTRACT

A quantified belief propagation (QBP) algorithm receives as input an existentially quantified boolean formula (QBF) of existentially quantified boolean variables, universally quantified variables, and boolean operators. A tripartite graph is constructed, and includes (i) there-exists nodes that correspond to and represent the existentially quantified variables, (ii) for-all nodes that correspond to and represent the universally quantified variables, and (iii) sub-formula nodes that correspond to and represent sub-formulas of the QBF. A set of boolean values of the existentially quantified variables is found by (i) passing a first message from an arbitrary sub-formula node to an arbitrary for-all node, and (ii) in response, passing a second message from the arbitrary for-all node to the arbitrary sub-formula node.

BACKGROUND

Quantified belief propagation has many applications, such as in thefield of program synthesis, which involves discovering programs thatrealize specified user intent. Program synthesis can be useful inscenarios such as: enabling people with no programming background todevelop utility programs, helping regular programmers automaticallydiscover tricky or mundane details, program understanding, discovery ofnew algorithms, and even teaching. The program synthesis problem can bethought of as a search for a desired program over a given search space,starting from some given specification of user intent.

In several of the program synthesis examples listed above, the problemreduces to the problem of solving a QBF (Quantified Boolean Formulas)formula of the form ∃ x·∀ y·φ( x, y), where x and y are vectors ofvariables, and all the existential quantifiers in the formula occurbefore all the universal quantifiers. This naturally happens when thesearch space is that of straight-line programs, or the userspecification is in the form of input-output examples. Recently, it wasshown that even synthesis of loopy programs can be reduced to solvingQBF formulas after some reduction involving the use of templates.

The class of straight-line programs, or more generally, loop-freeprograms, parameterized by the set of operators or components used, canoften express a wide range of useful programs. Even though these typesof programs may not involve loops, they can be challenging tosynthesize. Moreover, the space of loop-free programs is vast and thenumber of possible programs is exponential in the number of componentsin the program. Brute-force search methods do not scale well for theseproblems.

There are two drawbacks in the current development approaches to solvingprogram synthesis problems. First, most of the abovementioned programsynthesis problems have necessitated development of new problem-specificalgorithms. These algorithms are based on a variety of approachesranging from exhaustive search, version space algebras, geneticprogramming, and SAT/SMT (Satisfaction/Satisfiability Modula Theories)solving. The second drawback is that these specialized algorithms do notscale well.

Described below are techniques to solve ∃*∀*QBF formulas (existentiallyand universally quantified QBF) using probabilistic inferencetechniques. While the techniques are useful for program synthesis, theyhave application beyond the program synthesis problem.

SUMMARY

The following summary is included only to introduce some conceptsdiscussed in the Detailed Description below. This summary is notcomprehensive and is not intended to delineate the scope of the claimedsubject matter, which is set forth by the claims presented at the end.

A quantified belief propagation (QBP) algorithm receives as input anexistentially quantified boolean formula (QBF) of existentiallyquantified boolean variables, universally quantified variables, andboolean operators. A tripartite graph is constructed, and includes (i)there-exists nodes that correspond to and represent the existentiallyquantified variables, (ii) for-all nodes that correspond to andrepresent the universally quantified variables, and (iii) sub-formulanodes that correspond to and represent sub-formulas of the QBF. A set ofboolean values of the existentially quantified variables is found by (i)passing a first message from an arbitrary sub-formula node to anarbitrary for-all node, and (ii) in response, passing a second messagefrom the arbitrary for-all node to the arbitrary sub-formula node.

Many of the attendant features will be explained below with reference tothe following detailed description considered in connection with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The present description will be better understood from the followingdetailed description read in light of the accompanying drawings, whereinlike reference numerals are used to designate like parts in theaccompanying description.

FIG. 1 shows a Decimate procedure.

FIG. 2 shows a factor graph for an example F_(φ).

FIG. 3 shows a belief propagation algorithm.

FIG. 4 shows a QFG (quantified factor graph).

FIG. 5 shows an algorithm for solving QBFs.

FIG. 6 shows a quantified belief propagation (QBP) algorithm.

FIG. 7 shows a benchmark table.

FIG. 8 shows programs synthesized by the algorithm in FIG. 5.

FIG. 9 shows program specifications.

FIG. 10 shows programs synthesized for the specifications in FIG. 9.

FIG. 11 shows comparative results.

FIG. 12 shows a computer.

DETAILED DESCRIPTION 1. Introduction

To aid understanding, a review will be provided regarding howprobabilistic inference, and more specifically belief propagation, isused to solve SAT formulas. An SAT formula ∃ x·φ( x), where φ( x) is aconjunction of formulae φ₁( x), φ₂( x), φ₃( x), . . . , φ_(m)( x), isrepresented as a factor graph, which is a bipartite graph (for anexample, see FIG. 2) with one part in the partition consisting of nodes(called variable nodes) to represent each of the variables in x, andanother part in the partition consisting of nodes (called factor nodes)to represent each component of the conjunction φ₁( x), φ₂( x), φ₃( x), .. . , φ_(m)( x). An edge is drawn from a factor node φ_(i) to a variablenode x_(j) if the function φ_(i) depends on variable x_(j).

Belief propagation is performed by iteratively sending messages acrosseach edge in the factor graph. A variable node sends a message x_(j) toa factor node φ_(i), indicating the probability with which x_(j) istrue, and a factor node φ_(i) sends a message x_(j) indicating therequirement φ_(i) has on the probability distribution of x_(j) for φ_(i)to be satisfied (for a formal presentation of this algorithm, seeSection 2, FIG. 3). The messages are repeated in rounds untilconvergence is reached (that is, the messages sent on every edge insuccessive rounds is the same). Upon convergence of belief propagation,the messages sent by each variable node x_(j) approximate the marginalprobability distribution of x_(j) conditioned on φ_(i) being true. Atechnique for SAT solving, called decimation, picks a variable x_(i) anda value u such that the marginal probability that x_(i)=u is the highest(among all variables in x, and among all values), sets x_(i) to u, andsimplifies the formula φ with this assignment, leading to a formula φ(x)|_(x) _(i) _(=u) in which variable x_(i) has been substituted withvalue u. Next, belief propagation is run on the reduced formula φ(x)|_(x) _(i) _(=u) to find the next variable x_(j) and value u′ with thehighest marginal, and this process repeats until all variables areassigned values. If the marginal computation is exact, which is the casefor acyclic factor graphs, decimation is a provably exact algorithm tosolve SAT. If the factor graph has cycles, than the marginals computedon convergence of belief propagation are approximate, and decimationbecomes a heuristic (without theoretical guarantees) that works well inpractice.

A new decimation based approach is described herein to solve ∃*∀*QBFformulas. The algorithm (to be referred to as QOOL!) works over a newkind of factor graph called a Quantified Factor Graph (QFG). QFGs aretripartite graphs (see FIG. 4 for an example), with one part in thepartition to represent factor nodes, one part in the partition torepresent existentially quantified nodes, and one part in the partitionto represent universally quantified nodes. Notable is the use of a newmessage passing algorithm over these QFGs. A significant rule in themessage passing algorithm is that universal nodes function as“reflectors”—that is, if they receive a message μ from a factor node,they simply send a complement message

μ back to the factor node. The complement message

μ complements the probabilities in μ. If μ(x)=

(p), the Bernoulli distribution with mean p, then

μ(x) is defined as the distribution

(1−p).

As discussed in Section 3 below (and in particular Theorem 1), thismodified message passing algorithm over a QFG for a ∃*∀*QBF φ results inthe same marginal probabilities as obtained by transforming φ to a SATformula φ (by expanding out the universal quantifiers, potentiallyexploding the size of the formula), building a factor graph for φ, andrunning decimation based probabilistic inference algorithms on thefactor graph for φ. The message passing approach herein avoids thepotential exponential blowup from the QFG for the formula φ to thefactor graph for φ, while still providing the same marginalprobabilities that would be obtained even if the exploded factor graphfor were constructed.

QOOL! can be used to solve arbitrary QBF formulas by using skolemizationto convert all existential variables into skolem functions, and by usingtemplates to select these skolem functions from a finite solution space.This process of skolemization and templatization converts an arbitraryQBF into ∃*∀*QBF form. Satisfiability of the transformed ∃*∀*QBF formulagives a sufficient condition for satisfiability of the original QBF. Incase the transformed ∃*∀*QBF is satisfiable (not all are satisfiable),the QOOL! algorithm generates witness functions (or models) for each ofthe existentially quantified variables. Model generation is anotheruseful feature of QOOL!, which is lacking in many QBF solvers. In thecontext of program synthesis, the generated models can be used toconstruct the synthesized programs. Thus, QOOL! is particularly suitablefor program synthesis applications.

2. Preliminaries

This section introduces notation and background information. Familiaritywith propositional Boolean formulae, which are logical formulae over theset of Boolean operators {

,

,

} and Boolean variables, will be assumed. Quantified Boolean Formulae(QBF) which are propositional Boolean formulae with existential anduniversal quantifiers, are defined as follows.

Definition 1. The existential quantification of a formula φ with respectto a variable x is defined as:

$\begin{matrix}{{\exists{x \cdot \varphi}}\overset{def}{=}{{\varphi\left( {x = 1} \right)} ⩔ {\varphi\left( {x = 0} \right)}}} & (1)\end{matrix}$The universal quantification of a formula φ with respect to a variable xis defined as:

$\begin{matrix}{{\forall{x \cdot \varphi}}\overset{def}{=}{{\varphi\left( {x = 1} \right)} ⩓ {\varphi\left( {x = 0} \right)}}} & (2)\end{matrix}$

Definition 2. A propositional Boolean formula is said to be satisfiableif there exists an assignment to its variables such that the formulaevaluates to true. Computing a satisfying assignment to a propositionalBoolean formula is an NP-complete problem. It will be assumed that a QBFhas the following prenex normal form.Q ₁ x ₁ . . . Q _(n) x _(n)·φ  (3)where φ is the matrix which is a quantifier-free propositional formulaover variables x_(i), 1≦i≦n, and where Q_(i)ε{∃, ∀}, 1≦i≦n is a prefixof quantifiers.

Every QBF in prenex normal form can be transformed to an equisatisfiableformula with only universally quantified variables via skolemization.Specifically, given a QBF Q₁z₁ . . . Q_(n)z_(n)·φ in prenex normal form,the process of skolemization replaces every existentially quantifiedvariable y with a fresh function symbol ƒ_(y)(z₁, . . . , z_(m)) (calledthe skolem function), where the variables z_(i)ε{x_(i), . . . , x_(n)},1≦i≦m, are universally quantified such that ∀y occurs in the scope oftheir quantifiers.

Example 1

Consider the QBF ∀x·∃y·p(x)

q(x, y). The skolemized formula is ∀x·p(x)

q(x, ƒ_(y)(x)), where ƒ_(y) is the skolem function for the existentiallyquantified variable y.

Definition 3. Let φ be a QBF formula, and let {circumflex over (φ)} beits skolemized representation with only universally quantifiedvariables. A QBF formula φ is satisfiable if and only if there existskolem functions such that the formula {circumflex over (φ)} issatisfied for all values of the universally quantified variables.Determining whether a QBF is satisfiable or not is a PSPACE-completeproblem.

The availability of templates for skolem functions will be assumed. Atemplate restricts the space of possible skolem functions in order tomake the search for a skolem function more tractable. This naturallyfits with the objective of using a QBF solver to synthesize programs.

Example 2

Consider the skolem function ƒ_(y)(x) from Example 1. By assuming atemplate ƒ_(y) (x)ε{x,

x}, the search space for ƒ_(y) can be reduced and a Boolean variable opcan be used to specify which of the two functions is selected forƒ_(y)(x). This reduction can be significant for large QBFs. Furthermore,with this template, the skolemized formula now takes the following form:∃op·∀x·p(x)

q(x,(op→x

op→

x))

The next example illustrates skolemization with templates for a QBF withan alternation depth of three together with its satisfying model.

Example 3

Consider the following QBF.∀x·∃y·∀z·∃t·(x

y)

(

x

y)

(

t

x

z)

(t

x)

(t

z)By assuming templates ƒ_(y)(x)={x,

} and ƒ_(t)(x, z)={x

z,

x

z} for the skolem functions corresponding to variables y and trespectively, and variables op₁ and op₂ to specify which of thesefunctions is selected for ƒ_(y) and ƒ_(t) respectively, the followingformula is obtained upon skolemization.∃op ₁ ·∃op ₂ ·∀op ₂ ·∀x·∃z·(x

((op ₁ →

x)

(

op ₁ →x)))

(

x

((

op ₁ →

x)

(op ₁ →x)))

(((

op ₂ =x

z)

(op ₂ →

x

z))

x

z)

(((op ₂ →x

z)

(

op ₂ →

x

z))

x)

(((op ₂ →x

z)

(

op ₂ →

x

z))

z)This QBF is satisfiable with a model defined by the skolem functionsƒ_(y)(x)=

x and ƒ_(t)(x,z)=x

z.

Thus, it can be seen that any QBF in prenex normal form under thetemplate restriction and skolemization reduces to a formula with aprefix of quantifiers consisting of existentially quantified variablesfollowed by universally quantified variables. This class of QBFs will bereferred to as ∃*∀* formulae. The design of an algorithm for checkingthe satisfiability of ∃*∀* formulae is described in Section 3.

Note that using skolemization and template restriction producessolutions to the existentially quantified variables of the resulting∃*∀* formulae produces instantiations for the skolem functions. Forinstance, the valuations to op₁ and op₂ in the formula for Example 3(which is produced by our algorithm in Section 3), can be used to fixthe skolem functions that are chosen for ƒ_(y) and ƒ_(t) respectively.

The algorithm to solve QBF formulae of the form ∃ x·∀ y·φ( x, y) doesnot require that the matrix ( x, y) be in conjunctive normal form (CNF).It merely requires that ( x, y) be expressed as a conjunction ofsubformulas, where each subformula is not necessarily a clause. This isnotable due to the following two reasons.

First, the skolemizing transformations followed by templatization doesnot produce a CNF matrix (as illustrated in Examples 2 and 3 above). Theresulting non-CNF matrix could be converted to CNF, but that is anexpensive computation. Alternatively, temporary variables can be used toconvert the non-CNF matrix to CNF without exploding the size, but theresulting QBF will be of the form ∃*∀*∃, due to the addition oftemporary variables used in CNF conversion.

Second, if a QBF ∃ x·∀ yφ( x, y) is such that φ( x, y) is in CNF, then,it can be reduced to SAT by deleting all the variables y from each ofthe clauses in ( x, y). Deleting does not require ( x, y) to be in CNF.Such QBFs whose matrices are not necessarily in CNF naturally arise inprogram synthesis.

2.1 SAT Solving Via Probabilistic Inference

This section reviews a probabilistic inference algorithm called thebelief propagation (BP) algorithm in the context of SAT solving.

Consider a Boolean formula φ=

_(i=1) ^(k) φ_(i) that is a conjunction of k clauses φ_(i), 1≦i≦k,defined over n Boolean variables {x₁, . . . x_(n)}. The Boolean formulacan be expressed as a Boolean function:F _(φ):{0,1}^(n)→{0,1},where F_(φ)(x₁=u₁, . . . , x_(n))=u_(n))=1, if (u₁, . . . , u_(n)) is asatisfying assignment of the formula φ, otherwise, F_(φ)(x₁=u₁, . . . ,x_(n)=u_(n))=0. The Boolean function F_(φ) _(i) for a clause φ_(i)defined similarly. Thus, the Boolean satisfiability problem for thepropositional Boolean formula φ can be defined as follows.

Definition 4. Given a propositional Boolean formula φ=

_(i=1) ^(k) φ_(i) defined over n variables x₁, . . . , x_(n) and kclauses φ_(i), 1≦i≦k, does there exist an assignment (u₁, . . . ,u_(n))ε{0, 1}^(n) such that F_(φ)(u₁, . . . , u_(n))=1_(?)

FIG. 1 shows a Decimate 100 procedure. Belief propagation based SATsolvers employ decimation, as illustrated in the Decimate 100 procedure,to efficiently solve SAT problems. The Decimate shown in FIG. 1 takes aBoolean formula φ over variables {x₁, . . . , x_(n)} as input, andreturns a satisfying assignment σ: {x_(i), . . . , x_(n)} if φ issatisfiable. Decimate 100 is an incomplete procedure that is notguaranteed to either report a satisfying assignment or prove that theformula φ is unsatisfiable. Decimate 100 iteratively calls the procedureBP (described in Section 2.2) which is the belief propagation algorithmthat returns a tuple (x, u). Essentially, BP (F_(φ)(X)) returns the“(variable, value)” pair (x, u) with the highest marginal probability(the strongest “belief” about the formula). Informally, the marginalprobability p_(i)(x_(i)=u) denotes the fraction of solutions of theformula where the variable x is set uε{0, 1} and is defined as follows.

Definition 5. Let F_(φ)(x₁, . . . , x_(n)) be the Boolean function for aBoolean formula φ. The marginal distribution p_(i)(x_(i)) is defined asfollows.

$\begin{matrix}{{p_{i}\left( x_{i} \right)} = {\frac{1}{Z}{\sum\limits_{x_{1}}\mspace{20mu}{\ldots\mspace{14mu}{\sum\limits_{x_{i} - 1}\;{\sum\limits_{x_{i} + 1}\mspace{20mu}{\ldots\mspace{14mu}{\sum\limits_{x_{n^{t}}}\;{F_{\varphi}\left( {x_{1},\ldots\mspace{14mu},x_{n}} \right)}}}}}}}}} & (4)\end{matrix}$where Z=Σ_(x) ₁ _(. . . x) _(n) F_(φ)(x₁, . . . , x_(n)) is anormalization constant. Equation 4 can be succinctly written as

$\begin{matrix}{{{p_{i}\left( x_{i} \right)} = {\frac{1}{Z}{\sum\limits_{\sim {\{ x_{1}\}}}\;{F_{\varphi}\left( \;{x_{1},\ldots\mspace{14mu},x_{n}} \right)}}}}\;} & (5)\end{matrix}$Given (x, u), the formula φ is simplified in line 5 (where φ|_(x=u)denotes the formula φ with variable x set to value u), and this processis continued until all variables have been assigned values. In line 8,the assignment σ is checked to see if it is indeed a satisfyingassignment to φ. If so, Decimate 100 returns σ, else it returns FAIL.

Equation 5 is an expensive computation (in fact, computing thisexpression is a

P-complete problem), and belief propagation over factor graphs is awidely used practical algorithm to tackle this problem.

2.2 Factor Graphs and Belief Propagation

Factor graphs are graphical models that the belief propagation algorithmuses to efficiently compute marginal probabilities. Belief propagationand its variants take advantage of the structure of factor graphs inorder to significantly speed up computation of marginal distributions.Factor graphs and belief propagation over factor graphs will now beformally introduced.

Let φ=

_(i=1) ^(k) φ_(i) be a Boolean formula over variables {x₁, . . . x_(n)},and F_(φ)(x₁, . . . , x_(n)) Π_(i=1) ^(k)F_(φ) _(i) (X_(φ) _(i) ), X_(φ)_(i)

{x₁, . . . , x_(n)}, 1≦i≦n, be its corresponding Boolean function.

Definition 6. A factor graph is a bipartite graph that represents thefactorization of the Boolean function F_(φ). A factor graph has twotypes of nodes:

Variable nodes: one node for every variable x_(i); and

Factor nodes: one node for every function F_(φ) _(i) .

A variable node x is connected to factor node F if and only if x is aparameter of F.

Example 4

Consider the following Boolean formula:

$\begin{matrix}{{\varphi\left( {x_{1},x_{2},x_{3}} \right)} = {\underset{\underset{\varphi_{1}}{︸}}{\left( {x_{1} ⩔ x_{2}} \right)} ⩓ \underset{\underset{\varphi_{2}}{︸}}{\left( {{⫬ x_{1}} ⩔ x_{3}} \right)}}} & (6)\end{matrix}$The Boolean function F_(φ) for φ isF _(φ)(x ₁ , . . . , x _(n))=F _(φ) ₁ (x ₁ ,x ₂)·F _(φ) ₂ (x ₁ ,x₃)  (7)where

$\begin{matrix}{{F_{\varphi 1}\left( {x_{1},x_{2}} \right)} = \left\{ \begin{matrix}1 & {{{{if}\mspace{14mu} x_{1}} ⩔ x_{2}} = {true}} \\0 & {otherwise}\end{matrix} \right.} & (8) \\{{F_{\varphi 2}\left( {x_{1},x_{2}} \right)} = \left\{ \begin{matrix}1 & {{{{if}\mspace{14mu}{⫬ x_{1}}} ⩔ x_{3}} = {true}} \\0 & {otherwise}\end{matrix} \right.} & (9)\end{matrix}$

FIG. 2 shows the factor graph 120 for F_(φ). There is a variable node122 for each x_(i), 1≦i≦3, and a factor node 124 (F_(φ) _(i) ) for eachclause φ_(i), 1≦i≦2. There is an edge 126 between variable node x_(i)and factor node F_(φ) _(j) is x_(i) is a parameter of F_(φ) _(j) .

FIG. 3 shows the belief propagation algorithm 140 (BP) over a factorgraph. BP is an iterative algorithm that passes messages μ over theedges of the factor graph until convergence. Let G=(V, E) be the inputfactor graph with vertex set V and edge set E (for ease of exposition, Gand F_(ƒ) will b used interchangeably). Every message μ: {0, 1}=[0, 1]maps a variable x to a probability. In other words, the messages areBernoulli probability distributions. A Bernoulli probabilitydistribution will be denoted by

(p) with mean p. Essentially, a random variable with distribution

(p) is equal to 1 with probability p, and is equal to 0 with probability(1−p). In lines 2-4, the BP algorithm 140 initializes every message froma variable node to a factor node to the uniform distribution

${\mathcal{B}\left( \frac{1}{2} \right)}.$Because the satisfying assignments to the formula are unknown, thisuniform probability distribution represents the fact that there is noinformation—that is, all assignments are initially assumed to bepossible solutions. Lines 6-16 are executed until convergence—that is,until there are no changes to the messages passed along the edges of G.Essentially, there are two types of messages that are passed along theedges of G, which are defined next.

Factor-to-Variable message: This is defined as

$\begin{matrix}{{\mu_{f\rightarrow x}(x)} = {\alpha_{f,x}{\sum\limits_{\sim {\{ x\}}}\;\left( {{f\left( X_{f} \right)}{\prod\limits_{y \in {{{??}{(f)}}\backslash{\{ x\}}}}\;{\mu_{y\rightarrow f}(y)}}} \right)}}} & (10)\end{matrix}$where α_(ƒ,x) is a normalization constant so that μ_(ƒ→x)(x) is a validprobability distribution. Informally, the message μ_(ƒ→x)(x) that afactor ƒ sends to a variable x is the marginal function (over x) of thepointwise product² of all the μ_(y→ƒ)(x) that it receives from itsneighboring variable nodes (denoted by

(ƒ)) and the factor ƒ(X_(ƒ)) (where X_(ƒ)

V are the parameters of ƒ).

Variable-to-Factor message: This is defined as

$\begin{matrix}{{\mu_{x\rightarrow f}(x)} = {\alpha_{x,f}{\prod\limits_{h \in {{{??}{(x)}}\backslash{\{ f\}}}}\;{\mu_{h\rightarrow x}(x)}}}} & (11)\end{matrix}$where α_(x,ƒ) is a normalization constant so that μ_(x→ƒ)(x) is a validprobability distribution. Informally, the message μ_(x→ƒ)(x) that avariable x sends to a factor ƒ is the pointwise product of all themessages μ_(h→x)(x) that it receives from its neighboring factor nodes(denoted by N(ƒ)).

After convergence, lines 17-19 of the belief propagation algorithm 140compute the marginal distribution p_(i)(x_(i)) for every variable x_(i)defined as follows.

$\begin{matrix}{{p_{i}\left( x_{i} \right)} = {\alpha_{x_{i}}{\prod\limits_{f \in {{??}{(x_{i})}}}\;{\mu_{f\rightarrow x_{i}}\left( x_{i} \right)}}}} & (12)\end{matrix}$where x_(i) is a normalization constant such that p_(i)(x_(i)) is avalid probability distribution.

Finally, in line 20, the algorithm BP returns the variable x_(i) andvalue u with the highest marginal probability p_(i)(x_(i)=u). Thealgorithm BP computes the exact maximum marginal if and only if theinput factor graph G is acyclic. Otherwise, it computes an approximationto the marginal value—it is well known that these approximations tomarginal probabilities are very good in practice for many applications.In fact, it has been shown empirically that choosing maximum BPmarginals during decimation results in good approximations to the actualmarginal distributions.

Example 5

Consider the propositional Boolean formula in Example 4. The factorgraph for this formula is shown in FIG. 2. The flow of messagesgenerated by the BP algorithm over this factor graph will now beexplained.

Initialization: All variable-to-factor messages are initialized to theuniform Bernoulli distribution

${\mathcal{B}\left( \frac{1}{2} \right)}.$

${{\mu_{x_{1}\rightarrow F_{\varphi_{1}}}\left( x_{1} \right)} = {\mathcal{B}\left( \frac{1}{2} \right)}},\mspace{14mu}{{\mu_{x_{1}\rightarrow F_{\varphi_{2}}}\left( x_{1} \right)} = {\mathcal{B}\left( \frac{1}{2} \right)}}$${{\mu_{x_{2}\rightarrow F_{\varphi_{1}}}\left( x_{2} \right)} = {\mathcal{B}\left( \frac{1}{2} \right)}},\mspace{14mu}{{\mu_{x_{3}\rightarrow F_{\varphi_{2}}}\left( x_{3} \right)} = {\mathcal{B}\left( \frac{1}{2} \right)}}$

Iterations: Next, the messages are computed iteratively using Equations10 and 11. In every step, the algorithm shows only messages that aredifferent from the previous step and those that are required to showconvergence (achieved in Step 3).

Step 1:

${{\mu_{F_{\varphi_{1}}\rightarrow x_{1}}\left( x_{1} \right)} = {\mathcal{B}\left( \frac{2}{3} \right)}},\mspace{14mu}{{\mu_{F_{\varphi_{1}}\rightarrow x_{2}}\left( x_{2} \right)} = {\mathcal{B}\left( \frac{2}{3} \right)}}$${\mu_{F_{\varphi_{2}}\rightarrow x_{1}}\left( x_{1} \right)} = {\mathcal{B}\left( \frac{1}{3} \right)}$Step 2:

${{\mu_{x_{1}\rightarrow F_{\varphi_{2}}}\left( x_{1} \right)} = {\mathcal{B}\left( \frac{2}{3} \right)}},\mspace{14mu}{{\mu_{x_{1}\rightarrow F_{\varphi_{1}}}\left( x_{1} \right)} = {\mathcal{B}\left( \frac{1}{3} \right)}}$${{\mu_{F_{\varphi_{2}}\rightarrow x_{3}}\left( x_{3} \right)} = {\mathcal{B}\left( \frac{3}{4} \right)}},\mspace{14mu}{{\mu_{F_{\varphi_{1}}\rightarrow x_{2}}\left( x_{2} \right)} = {\mathcal{B}\left( \frac{3}{4} \right)}}$

Marginal computation: After convergence, the marginal distributions arecomputed using Equation 12.

${p_{1}\left( x_{1} \right)} = {{{\mu_{F_{\varphi_{1}\longrightarrow x_{1}}}\left( x_{1} \right)} \cdot {{\mu_{F}}_{\varphi_{2}\longrightarrow x_{1}}\left( x_{1} \right)}} = {\mathcal{B}\left( \frac{1}{2} \right)}}$${p_{2}\left( x_{2} \right)} = {{\mu_{F_{\varphi_{1}\longrightarrow x_{2}}}\left( x_{2} \right)} = {\mathcal{B}\left( \frac{3}{4} \right)}}$${p_{3}\left( x_{3} \right)} = {{\mu_{F_{\varphi_{2}\longrightarrow x_{3}}}\left( x_{3} \right)} = {\mathcal{B}\left( \frac{3}{4} \right)}}$

Highest marginal: There are two highest marginal probabilities

${{p_{2}\left( {x_{2} = 1} \right)} = {{p_{3}\left( {x_{3} = 1} \right)} = \frac{3}{4}}},$and thus the algorithm BP returns one of the pairs (x₂, 1), (x₃, 1).Indeed, the fraction of solutions of the formula φ(x₁, x₂, x₃) wherex₂=1 and x₃−1 is ¾.

Assume that BP returns (x₂, 1). This result is used by the Decimate 100procedure (in line 5) to simplify the Boolean formula φ(x₁, x₂, x₃) tothe Boolean formula {circumflex over (φ)}(x₁, x₃)=

x₁νx₃—this process is continued iteratively by calling BP on the factorgraph G_({circumflex over (φ)}) for the simplified formula {circumflexover (φ)}(x₁, x₃) until all variables have been assigned values.

The QOOL! Algorithm

This section describes the QOOL! algorithm for solving ∃*∀*QBFs. First,a new class of factor graphs is defined that represent QBFs and a newalgorithm to efficiently compute marginals over these graphs.

Definition 7. A quantified factor graph (QFG) is a tripartite graph thatrepresents the factorization of the Boolean function F_(φ) for thematrix φ of a QBF. A QFG has three types of nodes:

∃ Variable nodes: one node for every existentially quantified variable.

∀ Variable nodes: one node for every universally quantified variableu_(i).

Factor nodes: one node for every function F_(φ) _(i) in F_(φ).

A variable node x is connected to factor node F if and only if x is aparameter of F.

Example 6

Consider the following QBF.

$\begin{matrix}{\exists{x_{2}{x_{3} \cdot {\forall{x_{1} \cdot \underset{\underset{\varphi}{︸}}{\underset{\underset{\varphi_{1}}{︸}}{\left( {x_{1} ⩔ x_{2}} \right)} ⩓ \underset{\underset{\varphi_{2}}{︸}}{\left( {{⫬ x_{1}} ⩔ x_{3}} \right)}}}}}}} & (13)\end{matrix}$FIG. 4 shows the QFG 160 for this QBF. Both x₂ and x₃ are existentiallyquantified nodes, whereas x₁ is a universally quantified node (shown bya diamond shaped node). As shown in FIG. 4, there is a factor node forevery clause in the matrix of the QBF. There is an edge between variablenode x_(i) and factor node F_(φ) _(j) if x_(i) is a parameter of F_(φ)_(j) .

FIG. 5 shows the QOOL! algorithm 180 for solving QBFs. The QOOL!algorithm 180 takes a QBF φ as input and first converts it into askolemized QBF with matrix φ in line 1 (via the call to the procedureSkolemize which implements the skolemization process discussed inSection 2). Note that φ is a conjunction of propositional formulae thatare not necessarily clauses. Let E={e₁, . . . , e_(m)} and U={u₁, . . ., u_(n)} respectively be the set of existentially and universallyquantified variables in φ. As with the procedure Decimate described inSection 2.1, QOOL! is also an incomplete procedure. QOOL! iterativelycalls the procedure QBP (defined in Section 3.1) which is beliefpropagation over QFGs. The procedure QBP operates over an input QFG andreturns the (existentially quantified variable, value) pair (e, u) withthe highest marginal probability.

Given (e, u), the QBF φ is simplified in line 8, and this process iscontinued until all existentially quantified variables have beenassigned values. In line 12, the assignment σ is checked to see if it isindeed a satisfying assignment to the QBF φ (this check requires a callto a SAT solver). If so, QOOL! returns σ, else it returns FAIL.

3.1 Quantified Belief Propagation

FIG. 6 shows the quantified belief propagation (QBP) algorithm 200. TheQBP algorithm 200, like BP, is an iterative algorithm that passesmessages μ over the edges of the QFG until convergence. Let G=((V∪U), E)be the input QFG with existential vertex set V, universal vertex set Uand edge set E. Every message μ: {0, 1}→[0, 1] maps a variable xεV∪U toa probability. In lines 2-4, the algorithm QBP initializes every messagefrom a variable node to a factor node to the uniform distribution

${\beta\left( \frac{1}{2} \right)}.$Lines 6-19 are executed until convergence. Specifically, there are threetypes of messages that are passed along the edges of G, which aredescribed next.

Factor-to-Variable message: This is similar to the BP algorithm and isdefined as

$\begin{matrix}{{\mu_{f\longrightarrow x}(x)} = {\alpha_{f,x}{\sum\limits_{\sim{\{ x\}}}\left( {{f\left( X_{f} \right)}{\prod\limits_{y \in {{{??}{(f)}}\backslash{\{ x\}}}}{\mu_{y\longrightarrow f}(y)}}} \right)}}} & (14)\end{matrix}$Existential Variable-to-Factor message: This is similar to the BPalgorithm and is defined as

$\begin{matrix}{{\mu_{x\longrightarrow f}(f)} = {\alpha_{x,f}{\prod\limits_{h \in {{{??}{(x)}}\backslash{\{ f\}}}}{\mu_{h\longrightarrow x}(x)}}}} & (15)\end{matrix}$

Universal Variable-to-Factor message: This is defined asμ_(x→ƒ)(x)=

_(ƒ→x)(x)  (16)where the

operator is overloaded to work over random variables. If a randomvariable Y is distributed according to the probability distributionB(p), then the random variable

Y is distributed according to

(1−p). Essentially, a universal node acts as a “reflector” and sendsback the negation of the messages that it receives from its neighboringfactor nodes.

After convergence, lines 20-22 compute the marginal distributionp_(i)(x_(i)) for every variable x_(i) defined as follows.

$\begin{matrix}{{p_{i}\left( x_{i} \right)} = {\alpha_{x_{i}}{\prod\limits_{f \in {{??}{(x_{i})}}}{\mu_{f\longrightarrow x_{i}}\left( x_{i} \right)}}}} & (17)\end{matrix}$where x_(i) is a normalization constant such that p_(i)(x_(i)) is avalid probability distribution.

Finally, in line 23, the algorithm QBP returns the variable x and valueu with the highest marginal probabilityp _(i)(x _(i) =u).

Any QBF ∃ x·∀ y·φ( x, y) can be converted into a propositional Booleanformula ∃ x·∀ y·φ( x, y)( x, y), by iteratively using Equation 2 foreach universally quantified variable. Of course, any QBF can be expandedin this manner to a propositional Boolean formula which can then bechecked for satisifiability. Unfortunately, this procedure, in general,is likely to result in a blow-up of the propositional Boolean formulaunless PSPACE=NP. The following theorem shows that QBP over a QFG for aQBF is equivalent to BP over a factor graph for the expanded QBF.

Theorem 1. Let ∃ x·∀ y·φ( x, y) be a QBF and let G_(φ) be itscorresponding QFG. Let ∃ x·∃ y·φ( x, y) be the expanded version of theQBF. Let G_(φ) be its corresponding factor graph. Then the marginalprobabilities for existentially quantified variables in G_(φ) computedby QBP (G_(φ)) are equal to those computed by BP (G_(φ)).

Proof: to show that operationally QBP(G_(φ)) is essentially the same asBP(G_(φ))—that is, the marginal functions for existentially quantifiedvariables computed by QBP(G_(φ)) and BP(G_(φ)) are the same, let X={x₁ .. . , x_(m)} and Y={y₁, . . . , y_(n)}respectively be the set ofexistentially and universally quantified variables in the QBF.

The theorem will first be proven for QBFs with exactly one universallyquantified variable, that is, Y={y} and the corresponding QBF is ∃x·∀y·φ( x, y). The expanded QBF is the propositional Boolean formula ∃x·∃y·φ( x, y), φ( x, y)=φ( x, y)

φ( x,

y).

Let {F_(i)}_(1≦i≦k) be the set of factors in the QFG G_(φ) which have yas a parameter. Without loss of generality, it will be assumed that thisset is not empty. The factor graph G_(φ) consists of the factors{F_(i)}_(1≦i≦k) as well as additional factors {_(i)}_(1≦i≦k), where each

is the factor F_(i) with the universally quantified variable y negated.

First the following claim will be proven by induction on the number ofiterations n of the algorithm BP(G_(φ)).

Claim 1. In every iteration of BP(G_(φ)),μF _(i→y)(y)=

μ

_(→y)(y)=

μ_(y→F) _(i) (y)for i, 1≦i≦k.

Note that μ_(F) _(i→y) (y)=

μ_(y→F) _(i) (y) corresponds exactly to the case in QBP (G_(φ)) wherethe universally quantified variable y reflects its received message fromthe factor F_(i) in the QFG G_(φ) (given by Equation 16). Assume thatthe claim holds for all iterations less than n. Therefore, at the end ofiteration n−1, it is assumed that there are the following messages foreach i, 1≦i≦k,μ_(F) _(i) _(→y) ^((n−1))(y)=β(p _(i))  (18)

$\begin{matrix}{{\mu_{F_{i}^{⫬ y}\rightarrow y}^{({n - 1})}(y)} = {\mathcal{B}\left( {1 - {pi}} \right)}} & (19) \\{{\mu_{y\rightarrow F_{i}}^{({n - 1})}(y)} = {\mathcal{B}\left( {1 - {pi}} \right)}} & (20)\end{matrix}$where p_(i)ε(0, 1), 1≦i≦k.

From Equation 11:

$\begin{matrix}{{{\mu_{y\rightarrow F_{i}}^{(n)}(y)} = \alpha_{y}},{F_{i} \cdot {\prod\limits_{j \neq i}\;{{\mu_{F_{j}\rightarrow y}^{({n - 1})}(y)} \cdot {\prod\limits_{j}\;{\mu_{F_{j}^{⫬ y}\rightarrow y}^{({n - 1})}(y)}}}}}} & (21)\end{matrix}$This can be rewritten as

$\begin{matrix}{{{{\mu_{y\longrightarrow F_{i}}^{(n)}(y)} = {{\mu_{F_{i}^{⫬ y}\longrightarrow y}^{({n - 1})}(y)} \cdot \alpha_{y}}},{F_{i} \cdot {\prod\limits_{j \neq i}{{\mu_{F_{i}\longrightarrow y}^{({n - 1})}(y)} \cdot {\mu_{F_{i}^{⫬ y}\longrightarrow y}^{({n - 1})}(y)}}}}}{and}} & (22) \\{{\mu_{y\longrightarrow F_{i}}^{(n)}(y)} = {{{\mu_{F_{i}^{⫬ y}\longrightarrow y}^{({n - 1})}(y)} \cdot {\prod\limits_{j \neq i}{{\mathcal{B}\left( {1 - p_{i}} \right)}{\mathcal{B}\left( p_{i} \right)}}}} = {\mu_{F_{i}^{⫬ y}\longrightarrow y}^{({n - 1})}(y)}}} & (23)\end{matrix}$

Equation 23 follows from the fact that for any pε(0, 1)

${{\mathcal{B}(p)} \cdot {\mathcal{B}\left( {1 - p} \right)}} = {\mathcal{B}\left( \frac{1}{2} \right)}$(after suitable normalization to ensure that the result is a probabilitydistribution).

$\begin{matrix}{{{{Since}\mspace{14mu}{\mu_{F_{i}^{⫬ y}\rightarrow y}^{(n)}(y)}} = {{\mu_{F_{i}^{⫬ y}\rightarrow y}^{({n - 1})}(y)} = {\mathcal{B}\left( {1 - {pi}} \right)}}},{{from}\mspace{14mu}{Equation}\mspace{14mu} 23\text{:}}} & (24) \\{{\mu_{y\rightarrow F_{i}}^{(n)}(y)} = {{\mu_{F_{i}^{⫬ y}\rightarrow y}^{(n)}(y)} = {\mathcal{B}\left( {1 - {pi}} \right)}}} & \;\end{matrix}$From Equation 11:

$\begin{matrix}{{{\mu_{y\longrightarrow F_{i}^{⫬ y}}^{(n)}(y)} = \alpha_{y}},{F_{i}^{⫬ y} \cdot {\prod\limits_{j \neq i}{{\mu_{F_{j}^{⫬ y}\longrightarrow y}^{({n - 1})}(y)} \cdot {\prod\limits_{j}{\mu_{F_{j}^{⫬ y}\longrightarrow y}^{({n - 1})}(y)}}}}}} & (25)\end{matrix}$This can be rewritten as

$\begin{matrix}{{{\mu_{y\longrightarrow F_{i}^{⫬ y}}^{(n)}(y)} = {{\mu_{F_{i}\longrightarrow y}^{({n - 1})}(y)} \cdot \alpha_{y,F_{i}^{⫬ y}} \cdot {\prod\limits_{j \neq i}{{\mu_{F_{i}\longrightarrow y}^{({n - 1})}(y)} \cdot {\mu_{F_{i}^{⫬ y}\longrightarrow y}^{({n - 1})}(y)}}}}}{and}} & (26) \\{{\mu_{y\longrightarrow F_{i}^{⫬ y}}^{(n)}(y)} = {{{\mu_{F_{i}\longrightarrow y}^{({n - 1})}(y)} \cdot {\prod\limits_{j \neq i}{{\mathcal{B}\left( {1 - p_{i}} \right)}{\mathcal{B}\left( p_{i} \right)}}}} = {{\mu_{F_{i}\longrightarrow y}^{({n - 1})}(y)}.}}} & (27) \\{{{{{Since}\mspace{14mu}{\mu_{F_{i}\longrightarrow y}^{(m)}(y)}} = {{\mu_{F_{i}\longrightarrow y}^{({n - 1})}(y)} = {\mathcal{B}\left( p_{i} \right)}}},{{from}\mspace{14mu}{Equation}\mspace{14mu} 27\text{:}}}{{\mu_{y\longrightarrow F_{i}^{⫬ y}}^{(n)}(y)} = {{\mu_{F_{i}\longrightarrow y}^{(n)}(y)} = {\mathcal{B}\left( p_{i} \right)}}}} & (28)\end{matrix}$

From Equation 24, Equation 28 and the fact

(p_(i))=

(1=p_(i)), the induction hypothesis is proven for iteration n.

$\begin{matrix}{{\mu_{F_{i}\rightarrow y}^{(n)}(y)} = {{⫬ {\mu_{F_{i}^{⫬ y}\rightarrow y}^{(n)}(y)}} = {⫬ {\mu_{y\rightarrow F_{i}}^{(n)}(y)}}}} & (29)\end{matrix}$From the initialization step in BP(G_(φ)) where all variable-to-factormessages are set to

${\mathcal{B}\left( \frac{1}{2} \right)},$the base case for the induction also follows and the claim follows.

Since the equations for QBP(G_(φ)) and BP(G_(φ)) are identical forexistentially quantified variables, in order to prove equivalence ofthese algorithms, Equation 16 will be proven for the universallyquantified variable y, which is exactly the statement of Claim 1.Therefore, QBP(G_(φ)) is equivalent to BP(G_(φ)) with restriction thatY={y}.

Consider the QBF with Y={y₁, . . . , y_(n)}. Now for any arbitrary yεY,let z be the sequence y with the universally quantified variable yremoved. Then, the following holds (from Equation 2)φ( x , y )=φ( x , z ,y)

φ( x , z ,

y)  (30)

Given this transformation, Claim 1 holds by treating y as the onlyuniversally quantified variable in the proof of the claim, and this canbe done for every yεY. Thus Claim 1 holds for the QBF with Y={y₁, . . ., y_(n)} and the theorem follows.

The essence of Theorem 1 is that QBP(G_(φ)) smartly performs BP(G_(φ))in a “quantification” aware manner—it naturally takes care ofuniversally quantified variables without expanding them out, thusproviding an avenue for scalability.

Example 7

Consider the QBF in Example 6. The QFG for this formula is shown in FIG.4. The flow of messages generated by the QBP algorithm over this factorgraph is as follows.

Initialization: All variable-to-factor messages are initialized to theuniform Bernoulli distribution

${\mathcal{B}\left( \frac{1}{2} \right)}.$

${{\mu_{x_{1}\longrightarrow F_{\varphi_{1}}}\left( x_{1} \right)} = {\mathcal{B}\left( \frac{1}{2} \right)}},{{\mu_{x_{1}\longrightarrow F_{\varphi_{2}}}\left( x_{1} \right)} = {\mathcal{B}\left( \frac{1}{2} \right)}}$${{\mu_{x_{2}{—F}_{\varphi_{1}}}\left( x_{2} \right)} = {\mathcal{B}\left( \frac{1}{2} \right)}},{{\mu_{x_{3}\longrightarrow F_{\varphi_{2}}}\left( x_{3} \right)} = {\mathcal{B}\left( \frac{1}{2} \right)}}$

Iterations: Next, the messages are computed iteratively using Equations14, 15 and 16. In every step, only shown are messages that are differentfrom the previous step and those that are required to show convergence(achieved in Step 3).

Step 1:

${{\mu_{F_{\varphi_{1}}\longrightarrow x_{1}}\left( x_{1} \right)} = {\mathcal{B}\left( \frac{2}{3} \right)}},{{\mu_{F_{\varphi_{2}\longrightarrow x_{1}}}\left( x_{1} \right)} = {\mathcal{B}\left( \frac{1}{3} \right)}}$Step 2:

${{\mu_{x_{1}->F_{\varphi_{1}}}\left( x_{1} \right)} = {\mathcal{B}\left( \frac{1}{3} \right)}},{{\mu_{x_{1}->F_{\varphi_{2}}}\left( x_{1} \right)} = {\mathcal{B}\left( \frac{2}{3} \right)}}$Step 3:

${{\mu_{F_{\varphi_{1}}->x_{2}}\left( x_{2} \right)} = {\mathcal{B}\left( \frac{3}{4} \right)}},{{\mu_{F_{\varphi_{2}}->x_{3}}\left( x_{3} \right)} = {\mathcal{B}\left( \frac{3}{4} \right)}}$

Marginal computation: After convergence, the marginal distributions arecomputed using Equation 17

${p_{1}\left( x_{1} \right)} = {{\mu_{F_{\varphi_{1}}}->{{{x_{1}\left( x_{1} \right)} \cdot \mu_{F_{\varphi_{2}}}}->{x_{1}\left( x_{1} \right)}}} = {\mathcal{B}\left( \frac{1}{2} \right)}}$${p_{2}\left( x_{2} \right)} = {{\mu_{F_{\varphi_{1}}->x_{2}}\left( x_{2} \right)} = {\mathcal{B}\left( \frac{3}{4} \right)}}$${p_{3}\left( x_{3} \right)} = {{\mu_{F_{\varphi_{2}}->x_{3}}\left( x_{3} \right)} = {\mathcal{B}\left( \frac{3}{4} \right)}}$

Highest marginal: There are two highest marginal probabilities forexistentially quantified variables

${{p_{2}\left( {x_{2} = 1} \right)} = {{p_{3}\left( {x_{3} = 1} \right)} = \frac{3}{4}}},$and thus the algorithm QBP returns one of the (x₂, 1), (x₃, 1). Asexpected, QBP computes a marginal distribution of

$\mathcal{B}\left( \frac{1}{2} \right)$for the universally quantified variable x₁. Indeed, since a solution fora QBF is defined over all values of a universally quantified variable,the uniform distribution

$\mathcal{B}\left( \frac{1}{2} \right)$for x₁ is obtained.

Assume that QBP returns (x₂, 1). Analogous to the Decimate procedure,QOOL! uses this result (in line 8) to simplify the QBF to ∃·x₃·∀x₁·(

x₁

x₃)—this process is continued iteratively by calling QBP on the QFGG_({circumflex over (φ)}) for the simplified formula ∃x₃·∀x₁·(

x₁

x₃) until all the existential variables have been assigned values.

Example 8

Here is another example that illustrates the iterations of QBP for thefollowing QBF.

$\begin{matrix}{\exists{x_{2} \cdot {\exists{x_{3} \cdot {\forall{x_{1} \cdot {\underset{\underset{\varphi_{1}}{︸}}{\left( {x_{1}\bigvee x_{2}} \right)}\bigwedge\underset{\underset{\varphi_{2}}{︸}}{\left( {x_{1}\bigvee{⫬ x_{3}}} \right)}}}}}}}} & (31)\end{matrix}$The QFG for this formula is the same as the one shown in FIG. 4. Theflow of messages generated by the QBP algorithm over this factor graphwill now be illustrated.

Initialization: All variable-to-factor messages are initialized to theuniform Bernoulli distribution

${\mathcal{B}\left( \frac{1}{2} \right)}.$

${{\mu_{x_{1}->F_{\varphi_{1}}}\left( x_{1} \right)} = {\mathcal{B}\left( \frac{1}{2} \right)}},{{\mu_{x_{1}->F_{\varphi_{2}}}\left( x_{1} \right)} = {\mathcal{B}\left( \frac{1}{2} \right)}}$${{\mu_{x_{2}->F_{\varphi_{1}}}\left( x_{2} \right)} = {\mathcal{B}\left( \frac{1}{2} \right)}},{{\mu_{x_{3}->F_{\varphi_{2}}}\left( x_{3} \right)} = {\mathcal{B}\left( \frac{1}{2} \right)}}$

Iterations: Next, the messages are computed iteratively using Equations14, 15 and 16. In every step, the algorithm only shows messages that aredifferent from the previous step and those that are required to showconvergence (achieved in Step 3).

Step 1:

${{\mu_{F_{\varphi_{1}}->x_{1}}\left( x_{1} \right)} = {\mathcal{B}\left( \frac{2}{3} \right)}},{{\mu_{F_{\varphi_{2}}->x_{1}}\left( x_{1} \right)} = {\mathcal{B}\left( \frac{2}{3} \right)}}$Step 2:

${{\mu_{x_{1\;}->F_{\varphi_{1}}}\left( x_{1} \right)} = {\mathcal{B}\left( \frac{1}{3} \right)}},{{\mu_{x_{1}->F_{\varphi_{2}}}\left( x_{1} \right)} = {\mathcal{B}\left( \frac{1}{3} \right)}}$Step 3:

${{\mu_{F_{\varphi_{1}}->x_{2}}\left( x_{2} \right)} = {\mathcal{B}\left( \frac{3}{4} \right)}},{{\mu_{F_{\varphi_{1}}->x_{3}}\left( x_{3} \right)} = {\mathcal{B}\left( \frac{1}{4} \right)}}$

Marginal computation: After convergence, the marginal distributions arecomputed using Equation 17.

${p_{1}\left( x_{1} \right)} = {{{\mu_{F_{\varphi_{1}}->x_{1}}\left( x_{1} \right)} \cdot {\mu_{F_{\varphi_{2}}->x_{1}}\left( x_{1} \right)}} = {\mathcal{B}\left( \frac{1}{2} \right)}}$${p_{2}\left( x_{2} \right)} = {{\mu_{F_{\varphi_{1}}->x_{2}}\left( x_{2} \right)} = {\mathcal{B}\left( \frac{3}{4} \right)}}$${p_{3}\left( x_{3} \right)} = {{\mu_{F_{\varphi_{2}}->x_{3}}\left( x_{3} \right)} = {\mathcal{B}\left( \frac{1}{4} \right)}}$

Highest marginal: There are two highest marginal probabilities forexistentially quantified variables

${{p_{2}\left( {x_{2} = 1} \right)} = {{p_{3}\left( {x_{3} = 0} \right)} = \frac{3}{4}}},$and thus the algorithm QBP returns one of the pairs (x₂, 1), (x₃, 0).Like in the earlier example, QBP computes a marginal distribution of

$\mathcal{B}\left( \frac{1}{2} \right)$for the universally quantified variable x₁.

4. Experimental Evaluation

A prototype of QOOL! can be implemented in C# using the INFER.NETframework, a .NET library that provides APIs to probabilistic inferencealgorithms. The Z3 theorem prover is used to implement thesatisfiability check in line 12 of the QOOL! algorithm 180 shown in FIG.5. All experiments were performed on a system with 1.67 GHz Intel Core 2Duo processor and 2 GB RAM running Microsoft Windows Vista™.

We focus on the task of synthesizing the class of straight-lineprograms, or more generally, loop-free programs, parameterized by theset of operators/components used. This class covers a wide range ofuseful computations.

Every program synthesis instance considered is described by a logicalspecification that relates inputs and outputs together with a set ofbase components. Our objective is to synthesize a straight-line programthat satisfies the specification and that is constructed from a subsetof the base components. Known techniques can be used to convert programsynthesis instances to QBFs which are consumed by QOOL!. The model orsatisfying assignment to the QBF produced by QOOL! is the desiredsynthesized program.

4.1 Benchmarks

QOOL! was evaluated on a number of benchmarks from recent work onprogram synthesis. Benchmarks can be categorized as (a) Programssynthesis for integer arithmetic problems, (b) Programs synthesis forbitvector problems, and (c) Synthesizing strategies for games (inparticular, for the game NIM).

(a) Program Synthesis for Integer Arithmetic Problems:

These are benchmarks where the specification is an integer constraintover the program's input-output pairs. FIG. 7 shows a benchmarkspecification table 220. The table 220 shows the specifications for thebenchmarks I1, I2 and I3, which are described elsewhere. FIG. 8 showsprograms 240 synthesized by QOOL! for the specifications in FIG. 7. Forthese experiments, bit-blasting was used to convert the integerconstraints into a specification over Boolean variables. A set of basecomponents along with the specification for each component is taken asinput from the user. Using these components, an ∃*∀*QBF is constructed,which is solved using QOOL!.

(b) Program Synthesis for Bitvector Problems:

These are benchmarks for bitvector circuit synthesis problems. Thespecification 260 for these problems is given by the functional relationbetween the inputs and outputs of the circuit and is shown in FIG. 9.The programs 280 synthesized by QOOL! for these specifications are shownin FIG. 10. The functions used in these programs 280 have the standardsemantics as defined in SMTLIB QF BF logic. As in the case for integerarithmetic problems, bit-blasting was used to convert the bitvectorconstraints into a specification over Boolean variables.

(c) Synthesizing a Strategy for the NIM Game:

NIM is a two player game in which players take turns removing objectsfrom a set of distinct heaps, one or more objects from the same heap ata time. The player who removes the last object wins the game. NIM is nota fair game—given the right conditions, there is a strategy such thatthe player who starts the game cannot lose. The winning strategy for aplayer is to ensure that each move made leaves the game in aconfiguration where the bitwise XOR of the number of objects in eachheap is zero.

The problem of synthesizing a winning strategy for a version of NIMwhere there are only two heaps is considered. The specification for thisinstance of NIM is a ∃*∀* formula and is defined as follows.∀A,B·∃k·(XOR(A−k,B)=0)

(XOR(A,B−k)=0)Intuitively, this says that the target program should take two heapconfigurations A, B as input and return a move of size k such thatwinning strategy condition described above holds. With thisspecification, QOOL! is able to synthesize the following program whichrepresents the winning strategy.

4.2 Results

For performance evaluation, compare QOOL! is compared with the BPalgorithm over expanded QBF formulae with the universally quantifiedvariables eliminated. FIG. 11 shows results 300 of this comparison. Thebenchmark Q1 is the QBF formula shown in Example 3 and the benchmark Q2is a QBF formula taken from the Letz suite taken from QBFLIB. Thisformula has 10 existentially quantified variables, 10 universallyquantified variables and 36 clauses. As seen from FIG. 11, QOOL! clearlyoutperforms BP—the BP algorithm on the expanding formula times out onbenchmarks B5, B6 and Nim, whereas QOOL! completes successfully (wherethe time out is set to 30 minutes). It is also interesting to note thatthe SKETCH tool (v1.3.0) also times out on the benchmarks B5 and B6.

CONCLUSION

Program synthesis is an emerging area of active research, with severalapplications on the horizon. Just as SAT solvers have powered programverification tools over the past decade, QBF solvers can now powerprogram synthesis tools. In particular, if ∃*∀*QBF formulae can besolved in a scalable manner, then it is possible to reduce a large classof synthesis problems and arbitrary QBF formulae from other applications(such as model checking and game solving) to this fragment usingskolemization to witness existential variables, and using templates toinstantiate skolem functions from a finite state space.

Described herein is a new algorithm, QOOL!, based on a new beliefpropagation algorithm to solve ∃*∀*QBF formulae. Nodes can be treated ascorresponding to universally quantified variables in a special manner ingraphical models of these formulas, where the nodes negate incomingmessages and reflect them back to the sending nodes. It is proven above(Theorem 1) that this algorithm precisely captures the effect ofexpanding out the universal quantifiers, and produces the same marginalprobabilities as would be produced by such an expansion. Theimplementation of QOOL! is able to efficiently solve several synthesisproblems and other QBF problems from published literature, and in somecases is able to finish where previous tools have not been able tofinish.

Following are some examples of programs that can be solved orsynthesized with QOOL!: the set of components including arithmeticoperators and bitwise operators (bitvector algorithms); the set ofcomponents including basic editing commands available in a text-editorsuch as insert, locate, select and delete (text editing programs); theset of components including geometrical constructors such as ruler andcompass (geometric constructions); the set of components including thosethat the underlying decision procedure can reason about such as lineararithmetic operators and set operators (unbounded data typemanipulation); and API (application programming interface) callsequences.

FIG. 12 shows a computer 320 with which embodiments and featuresdiscussed above can be realized. The computer 320 has a processor 322cooperating with memory 324 and non-volatile storage 326. Theembodiments can be realized in the form of information stored involatile memory 324 or the non-volatile storage 326 (e.g., devicereadable media). This volatile memory 324 or the non-volatile storage326 is deemed computer-readable storage media, examples of which includemedia such as optical storage (e.g., compact-disk read-only memory(CD-ROM)), magnetic media, flash read-only memory (ROM), or any currentor future tangible means of storing digital information in a formconvenient for consumption by a processor. The stored information can bein the form of machine executable instructions (e.g., compiledexecutable binary code), source code, bytecode, or any other informationthat can be used to enable or configure computing devices to perform thevarious embodiments discussed above. Again, this information may bestored in various forms in random-access memory (RAM) and/or virtualmemory storing information such as central processing unit (CPU)instructions during execution of a program carrying out an embodiment,as well as non-volatile media storing information that allows a programor executable to be loaded and executed. The embodiments and featurescan be performed on any type of computing device, including portabledevices, workstations, servers, mobile wireless devices, and so on;computer 320 is merely an example.

The invention claimed is:
 1. A method performed by a computing devicecomprised of memory and a processor cooperating to perform the method,the method performed by the computing device comprising: executing, bythe processor, a quantified belief propagation (QBP) algorithm thatreceives as input, from the memory, an existentially quantified booleanformula comprised of existentially quantified boolean variables,universally quantified variables, and a boolean formula comprised ofsubformulas each comprising one or more boolean operators operating onone or more of the existentially quantified variables and/or one or moreof the universally quantified variables; constructing, in the memory, atripartite graph comprised of nodes comprising: (i) there-exists nodesthat correspond to and represent the existentially quantified variables,respectively, (ii) for-all nodes that correspond to and represent theuniversally quantified variables, and (iii) sub-formula nodes thatcorrespond to and represent the sub-formulas, respectively; finding aset of boolean values of the existentially quantified variables by (i)passing a first message from an arbitrary sub-formula node to anarbitrary for-all node, and (ii) in response, passing a second messagefrom the arbitrary for-all node to the arbitrary sub-formula node; andstoring the set of boolean values in the memory.
 2. A method accordingto claim 1, wherein the first message comprises a first probability,held by the arbitrary sub-formula node, of a boolean value of theuniversally quantified variable represented by the arbitrary for-allnode.
 3. A method according to claim 2, wherein the second messagecomprises a second probability computed from the first probability.
 4. Amethod according to claim 3, wherein the second probability is one minusthe first probability.
 5. A method according to claim 1, furthercomprising: executing an algorithm that uses the QBP algorithm to find asolution to a problem input to the algorithm, wherein the algorithmcomprises either: a program synthesis algorithm, or a model checkingalgorithm.
 6. A method according to claim 1, further comprising:executing a program synthesis algorithm that uses the QBP algorithm tofind a program that satisfies a program specification input to theprogram synthesis algorithm.
 7. A method according to claim 6, whereinthe using the QBP algorithm comprises generating the existentiallyquantified boolean formula from the program specification.
 8. Anapparatus storing information to enable a computing device to perform aprocess, the computing device comprising a processor and physicalstorage, the process comprising: performing, by the processor, a beliefpropagation algorithm by constructing a qualified factor graph stored inthe memory, the qualified factor graph comprising first nodes thatrepresent sub-formulas of a boolean formula, second nodes that representexistentially quantified boolean variables in the sub-formulas, andthird nodes representing universally quantified boolean variables in thesub-formulas; and finding and storing in the physical storage, by theprocessor, a solution to a boolean formula comprised of the sub-formulasby the processor passing messages from the third nodes to the firstnodes.
 9. An apparatus according to claim 8, the process furthercomprising passing a first message from a first node to a third node,and in response generating a second message based on the first messageand passing the second message from the third node to the first node,wherein the first message and the second message are passed within arepeating loop that passes messages between the first and the secondnodes and between the first and the third nodes, but not between thesecond and the third nodes, wherein the repeating loop repeats untilconvergence to the solution is detected.
 10. An apparatus according toclaim 9, wherein the second message cancels a probabilistic belief thatwould otherwise be propagated by the first message.
 11. An apparatusaccording to claim 8, wherein the messages from the third nodes to thefirst nodes reflect probabilistic inferences about values of theuniversally quantified boolean variables, wherein the solution to theboolean formula has specific true/false values for the existentiallyquantified boolean variables, and the solution values of theexistentially quantified boolean variables satisfy the boolean formularegardless of any specific boolean value of any of the universallyquantified variables.
 12. An apparatus according to claim 8, the processfurther comprising iteratively invoking the belief propagation algorithmfor each existentially quantified variable and each universallyquantified variable, where each iteration finds a true or false valuefor the corresponding universally quantified variable.
 13. An apparatusaccording to claim 8, where each message comprises a first probabilitythat a universally quantified variable represented by the third node istrue, and a second probability that the universally quantified variablerepresented by the third node is false.
 14. An apparatus according toclaim 13, wherein a cancellation message sent in response to one of themessages cancels a probabilistic effect of the one of the messages. 15.An apparatus according to claim 14, wherein the cancellation messagecomprises a third probability that the universally quantified variablerepresented by the third node is true and a fourth probability that theuniversally quantified variable represented by the third node is false.16. A computer comprised of a processor, memory, and non-volatilestorage, the non-volatile storage storing processor instructionscorresponding to the processor, the instructions, when executed by theprocessor, performing a process comprising: receiving, by the processor,and storing in the memory, a quantified boolean formula comprised ofvariables in the memory, including existentially quantified variables,universally quantified variables, and comprised of boolean operators inthe memory operating on the existentially quantified variables and theuniversally quantified variables; and finding, by the processor, andstoring in the memory, a solution comprising a set of boolean values ofthe respective existentially quantified variables that satisfy thequantified boolean formula, wherein the finding is performed byexecuting a belief propagation algorithm in the memory in a manner thattakes into account the universally quantified variables withoutexpanding the universally quantified variables, wherein the set ofboolean values are found based in part on inferred probabilities ofboolean values of the universally quantified variables.
 17. A computeraccording to claim 16, wherein the belief propagation algorithmautomatically reflects or negates probability inferences about theuniversally quantified variables.
 18. A computer according to claim 16,wherein the belief propagation algorithm, while iterating, checks forconvergence of a universally quantified variable, and also whileiterating passes messages from nodes representing sub-formulas of thequantified boolean formula to nodes representing the universallyquantified variables, which causes messages to be reflected back fromthe nodes representing the universally quantified variables to the nodesrepresenting the sub-formulas.
 19. A computer according to claim 16,wherein the belief propagation algorithm is invoked by a programsynthesis algorithm executing on the computer, the program synthesisalgorithm invoking the belief propagation algorithm to find a programthat satisfies a program specification provided as input to the programsynthesis algorithm.
 20. A computer according to claim 19, wherein theprogram specification specifies a bitvector program, a text-editingprogram, a geometric construction program, an unbounded data typemanipulation program, or a sequence of API (application programminginterface) calls.